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ABSTRACT 

This study was an empirical investigation of active student responding (ASR) utilizing a student response system [SRS] vs. 
single student questioning (SSQ) and no student responding in a graduate level special education class of 23 
participants. During the SRS condition, every participant responded to questions using remotes/clickers. During the SSQ 
condition, the instructor randomly called upon individual participants to vocally answer a question. During the control 
condition, no questions were asked of participants. An alternating treatments design was used to test the effects of the 
three conditions on the response accuracy to a short-answer quiz at the beginning of a session, and accuracy with which 
participants completed a task during which they must apply the information presented during the lecture. There was 
statistically significant difference in student performance on application tasks, but not statistically significant difference 
on quiz scores. The findings diverge from the results of other SRS studies and K-12 ASR studies, but support some college 
level studies. 
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INTRODUCTION 

In 21 st century instructors of colleges and universities are 
virtually bombarded with new and exciting forms of 
technology that ostensibly can enhance their teaching. 
From putting entire courses on-line, to hybrid courses, to 
on-line course management software, to podcasting 
lectures, to electronic discussion boards, to electronic 
response systems and many more, there are a myriad of 
ways to enhance instruction through technology. 
However, these technologies are no more or no less than 
an instructional tool that functions much in the same way 
as a chalk, a slate board, and an eraser, In other words, 
the purpose of the technology is to facilitate student 
learning of whatever content is taught. 

One of the technology enhancements that has been 
around for a long time is electronic student response 
system (SRS) (See Judson and Sawada, 2002 for an 
extensive historical overview of SRS). SRS is best defined as 
a technological response system or an input device 
based on a computer mediated wireless response 


system (Judson & Sawada, 2002). SRS is designed to 
encourage active participation in the classroom while 
creating an effective student-centered learning 
environment. SRS is usually comprised of three parts. The 
first part is the clicker, a small remote control device that 
allows students to press keys that correspond to answers, 
The key pressing may correspond to letters on keyboard or 
A-B-C-D multiple choice or True (or) False responses. The 
second part is a receiver designed to interact with the 
clicker and pick up responses encoded by students, In 
many SRS the student and instructor are given immediate 
feedback on the student's responses. Lastly, there is a 
computer program that the instructor uploads to his or her 
classroom computer to enable interface between a 
computer and the receiver and provides the instructor 
with presentation options and records all student 
responses in a database (Hatch, Jensen, & Moore, 2005). 
In general, research does support the use of SRS in 
college classrooms to enhance learning (Caldwell 2007; 
Hatch, Jensen, & Moore, 2005). Most studies have 
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reported positive results in terms of academic gains and 
personal satisfaction in the classroom (e.g., Martyn, 
2007). Stowell and Nelson (2007) compared SRS to 
response cards and standard lecture hand-raising during 
simulated introductory psychology classes. The results 
showed that students performed well during both the SRS 
and response card conditions, and performed 
significantly better than during hand-raising conditions. 
Other studies have shown that SRS systems can have 
positive effects on student attendance (Jackson & Tress, 
2003). Conversely, Bunce, VandenPlas, and Havanki 
(2006) found that students test scores were lower during 
SRS conditions as compared to WebCT conditions during 
chemistry classes. In addition, although they were 
generally positive of SRS, Hatch, Jensen and Moore (2005) 
reported frustration with technological glitches, Even 
though there is a significant and growing research base to 
support the use of SRS in college classrooms there are still 
many gaps in the literature. For example, little research 
has addressed how SRS affect applying lecture 
information to task specific activities. In addition, the 
authors found no studies that looked at the delayed recall 
of lecture information while utilizing SRS. Finally, most of the 
studies used undergraduate students as participants. 

The purpose of the present investigation was to address 
these deficiencies by investigating the effects of SRS, 
single student questioning, and no student questioning 
during a post-bachelorette special education teaching 
methods course. The study was designed to address the 
question, What are the differential effects, if any, of a 
student response system, single student questioning, and 
no questioning on participants' delayed recall, 
application of content, or response accuracy? 

Method 

Participants 

The participants were 23 graduate students enrolled in a 
graduate level special education class at a public 
university in the southern United States. Twenty-two 
participants were seeking a degree related to instruction 
and curriculum leadership, which includes special 
education. One participant was taking the course to 


obtain teaching certification only, Thirteen of the 
participants were African American, nine were 
Caucasian, and one was Hispanic. Thirteen were female, 
ten were male. The course was required for all the 
participants. The class met twice per week for 2 hours and 
50 minutes for 10 weeks. 

Dependent Variables 

The dependent variables measured were accuracy of 
delayed recall of lecture material, accuracy of 
application of lecture material, and frequency and 
accuracy of student responding. Delayed recall was 
assessed by giving the participants a short-answer, 5-item 
quiz at the beginning of the next class session. The quiz 
items required the participants to recall key points that 
were presented during the lecture. The quizzes were 
scored by the first author, who was the instructor for the 
class, and by a second observer. Data are reported in 
terms of number of items accurately answered. 
Application of lecture material was assessed by giving the 
participants' a task during which they must apply the 
information presented during the lecture. After the 
participants take the recall quiz, they were provided 
approximately 30 minutes to complete the application 
task (e.g., write an instructional objective). Tasks were 
evaluated using a rubric consisting of 3 points. In general 
a participant earned 3 points if they accurately 
completed the task, 2 points if a few elements of the task 
were missing, 1 point if several elements of the task were 
missing, and no points if all the elements were missing or if 
the task was incomplete, The first author evaluated the 
tasks, as did a second observer. Data are reported in 
terms of the number of points earned on the rubric. 
Accuracy of responding was measured by examining the 
participants' responses and the correct answer for one-to- 
one correspondence. Data were reported in terms of 
percentage of correct responses. Participants' responses 
were collected in vivo by the student response system 
during the SRS condition and by the instructor for the 
course during the SSQ condition. Accuracy of responding 
data was not collected during the control condition 
because questions were not asked to the participants. 
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Independent Variables 

Three experimental conditions were manipulated during 
the study: single student questioning (SSQ), student 
response system (SRS), and a control condition. In the SSQ 
condition, lecture information was discussed and 
presented using PowerPoint®. Periodically throughout the 
lecture the instructor randomly called upon individual 
participants to answer questions that were prepared prior 
to each class session. Questions were read aloud by the 
instructor and presented via PowerPoint®. A random 
number generator (http://www.random.org) generated 
list of numbers, corresponding to individual participants to 
be called upon, prior to each session. If a participant 
chose to "pass" the question, the instructor would provide 
the correct answer. Participants were informed at the 
beginning of the class that they would be asked to recall 
and apply the information after the lecture. An average of 
19 questions were asked during each session of this 
condition. 

In the SRS condition, the instructor conducted the lecture 
in the same manner as the previous condition. However, 
every participant used remotes/keypads with which they 
responded to questions posed by the instructor. Radio 
frequency clickers by Interwrite Personal Response 
System® (PRS) were utilized because the university had 
adopted this particular system. Interwrite PRS® provided 
30 seconds to select and enter an answer and displayed 
the percentage of responses to each choice after the 
question was closed (the computer no longer accepts 
responses for that question). An average of 19.75 
questions were asked during each session of this 
condition. 

The control condition consisted of the instructor 
presenting lecture information with PowerPoint® without 
asking participants questions. Participants were informed 
at the beginning of class that they would be asked to 
recall and apply the information afterthe lecture. 
Research Design 

Alternating treatments design was utilized during when 
the experimental conditions were alternated across 
sessions, An alternating treatment design is a repeated 


measures research design in which two or more 
treatments are rapidly alternated to determine their 
effects on behavior (Cooper, Heron, & Heward, 1987). 
Conditions were randomly assigned with no more than 2 
consecutive sessions utilizing the same condition. One 
experimental condition was conducted per class session, 
Procedures 

Each session consisted of a lecture with a corresponding 
PowerPoint® presentation during which the participants 
used guided notes. During the control condition, there 
were no opportunities for participants to respond to 
questions during the lecture. During SSQ and SRS 
conditions questions were presented during the lecture to 
which the participants responded. Feedback was 
provided after each question. If the majority of the 
responses were correct, the instructor praised or repeated 
the correct answer, If the responses were incorrect, the 
instructor repeated the correct answer with a brief 
explanation. Afterthe lecture in each session, participants 
were provided approximately 30 minutes to complete a 
task that required them to apply the information from the 
lecture, The participants were allowed to refer to their 
notes during the task. At the beginning of each session, 
participants answered a short-answer, 5-item quiz 
covering content from the lecture during the previous 
session. 

Reliability 

Participants were trained howto use the clickers to ensure 
that response errors were less likely due to technical 
difficulties with the clickers. Participants were asked to 
respond using the clickers to 8 questions with obvious 
correct answers (e.g., who is the president of the United 
States? A. Donald trump, B. George W. Bush, C. Martha 
Stewart). The mean response accuracy during training 
was 92,7% with a mean of 22,9 responses. 

Interrater agreement was calculated for 29% of the 
sessions. The instructor and an independent grader 
scored all quizzes and application tasks. The total 
agreement was calculated through dividing the number 
of items in which the graders agreed upon by the number 
of agreements plus disagreements and multiplying by 
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100, The mean agreement on delayed recall quizzes was 
98%. Mean agreement on the application tasks was 83%. 
Treatment integrity was calculated by a second observer 
by counting the number of questions asked during a 
session and the type of feedback provided for responses 
during 14% of sessions, A question and answer were 
scored accurate if the correct question was asked and 
the proper type of feedback was given, Feedback was 
considered accurate if a praise statement or restatement 
of the correct answer was provided based on the 
accuracy of the group or individual response. An 
incorrect answer in the majority of the group resulted in the 
instructor providing the correct answer and giving a brief 
explanation, Treatment integrity was calculated by 
counting the number of accurate questions and 
feedback completed divided by the number of accurate 
and inaccurate questions and feedback completed 
multiplied by 100. Treatment integrity was eighty-three 
percent across all observations. 

Results 

Participants scored slightly higher on quizzes during the 
SSQ condition [M = 4.8; SD = 0.28) than the SRS [M = 4.5; 
SD = 0,54) or control conditions [M = 4.5; SD = 0.68). 
Figure 1 displays the average score per quiz in each 
condition. A one-way analysis of variance (ANOVA) was 
used to assess the differences between the quiz score 
means in each condition for significance. The ANOVA did 
not reveal statistically significant difference in quiz scores 
for each condition (F (2, 66) = 2.96, p < 0.06). However, 
post hoc comparison using Tukey procedures revealed 
SSQ differed from SRS by 0,73 standard deviations and 
from the control condition by 0.65 standard deviations. 
While the findings may not be statistically significant, it 
could be argued that they are conceptually different. 
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Sessions 

Figure 1. Mean number of points earned on 
delayed 5-item quizzes during SRS and SSQ. 


Figure 2 shows the average application task score per 
session in each condition. Participants earned 
approximately the same amount of points on the tasks 
during the SSQ condition (M = 2,6; SD = 0,21) and SRS 
condition [M = 2.7; SD = 0.23). Both the SSQ and SRS 
conditions were superior to the control condition (M = 
2.3; SD = 0.58). An ANOVA of application task scores 
revealed there was a statistically significant difference 
between task scores in the conditions (F(2,65) = 4.78,p < 
0.01). A post hoc comparison using Tukey procedures 
revealed no statistical difference between SRS and SSQ 
application task scores (q = 1.25;p < 0.05) but did reveal 
statistically significant differences between SRS and 
control conditions (q = 5.02; p < 0.05) and SSQ and 
control conditions (q = 3.77; p < 0.05). There was a large 
effect size in favor of application task scores during the 
SRS condition when compared to the control (Cohen's 
d = 0.89) and also of the SSQ condition when compared 
to the control (Cohen's d= 0.67). 

Differences in response accuracy during SRS and SSQ 
were evaluated for significance using a T-test. There was 
no significant difference in response accuracy between 
the conditions [t (-0,13) = 2,02, p = ,89, two-tailed). 
Figure 3 depicts the average accuracy of responses 
during the SSQ and SRS conditions. Responses were more 
accurate during SSQ [M =78%; SD = 0.35%) than during 
the SRS condition [M = 77%; SD = 0.07). More total 
responses were made during the SRS condition [M = 21.5; 
Range = 20.4 - 22.5) than the SSQ condition (M =18; 
Range = 1 7-19). 

At the end of the study, participants (n = 22) completed a 
questionnaire that contains 12 social validity question to 
which they responded strongly agree, agree, disagree, 
strongly disagree, or not sure. Several items addressed 
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Figure 2, Mean number of points earned on 
application tasks during SRS and SSQ. 
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Figure 3. Mean accuracy of participant responses 
during SRS and SSQ. 

engagement and on-task behavior. Forty-five percent of 
the participants said that they paid better attention during 
SRS condition and 59% thought that they have 
participated more with SRS. However, participants said 
SRS aided with their on-task behavior (68%) and that there 
was more class participation during SRS (82%), Regarding 
the effect of SRS on learning, participants said that they 
had more time for questions and comments during SRS 
(50%), spent more time listening and thinking about 
concepts with SRS (64%), learned more with SRS (64%), 
and retained more information with SRS (59%). Similarly, 
45% of participants were not sure if their quiz scores were 
better when utilizing the SRS. As for personal preferences, 
91% of participants said that they had a positive 
experience with SRS. Seventy-seven percent of 
participants preferred SRS to SSQ and 86% recommend 
using SRS in future classes. Space was provided on the 
questionnaire for additional comments. There were 15 
additional comments. Generally the comments 
addressed four areas: elements of pacing such as 
feedback delay and intertrial interval, feedback on 
percent accuracy across all questions not just individual 
questions, closer relation between questions and quizzes, 
and use of SRS for assessment. 

Discussion and Conclusion 

In general, regardless of the conditions, participants 
performed well during this study. In terms of letter grades 
(i.e., what the students really care about), there was only 
an average of a difference of half a letter grade on quiz 
scores and application tasks between SRS and SSQ 
conditions. However, there was a full letter and a half 
grade difference between SRS and the control condition 
and a full letter grade difference between SSQ and the 
control conditions on application tasks. In other words, the 


differential effects between the SRS and SSQ 
technologies were minimal and both were better than no 
questioning at all. These results are congruent with Stowell 
and Nelson (2007) and similar to Shabani and Carr (2004) 
who investigated a low-tech form of active student 
responding. The results stand in contrast to those of 
Bunce, VandenPias and Havanki (2006). However, there is 
a major limitation to this study that must temper any 
conclusions. First, the researchers had to balance the 
need to increase the likelihood that students learned the 
material for the class with wanting to get "clean" results 
from this study. Therefore guided notes were chosen to 
use in all conditions. Guided notes are a standard 
technology used to increase active student responding 
during lectures. Guided notes are instructor prepared 
handouts that contain cures to write important 
information presented in class (See Heward, 1994 for a 
review of guided notes research). Studies have shown that 
guided notes alone increase college student's quiz 
scores (Austin, Lee, & Carr, 2004; Austin et al., 2002; 
Lazarus, 1993), Although the guided notes were optional 
almost all the students chose to use them and they were 
consistently provided across conditions. The use of 
guided notes may have confounded the results of this 
study because the students were given important 
information from which to study for the next class quizzes. 
This may be the reason for why test scores were not 
significantly different across all conditions, In addition, the 
low number of quiz questions could have impacted 
differences as well. A ceiling effects was seen on quiz 
scores in this study. A future study would be well served to 
include more questions than the present study did per 
quiz. An interesting topic for a future study may be to 
include higher order comprehension questions along with 
higher order application tasks. However, a limitation of the 
SRS technology is the ability of the clicker devices to allow 
students to produce sophisticated answers, which is 
discussed in the later section of this paper. 

A second limitation to the investigation was the method 
chosen to implement the SSQ conditions. Instead of 
allowing students to voluntarily answer questions or 
choosing students in a predictable manner the 
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researchers chose to use an indiscriminable contingency 
during SSQ conditions. Participants were randomly 
selected to answer questions. Although participants were 
allowed to "pass" the question, only one participant ever 
did this. The reason for choosing to use this method was it 
was more likely to keep participants "on their toes" during 
SSQ conditions (Brame, 2001), The researchers also 
wanted to provide every participant with an opportunity 
to respond at least once during SSQ. Yet, by doing so it 
might have inadvertently elevated participant learning. 
So, again the researchers had been trying to balance the 
dual goals of fostering learning and investigating 
teaching methods in the classrooms. Future investigators 
may want to try to disentangle aspects of the method that 
may yield clearer results. 

Finally, it is important to note that although the participants 
generally were favorable to the SRS system there were 
some complaints about the length of the response times 
(many thought that the response times were too long). 
Additionally, there was somewhat of a steep learning 
curve in learning to integrate the technology into the 
PowerPoint® presentations and in learning to transfer the 
data from the system to a spreadsheet. 

Despite the limitations, it is important to discuss the 
possible reasons why the two questioning conditions were 
better than the no questioning condition. The difference 
between the questioning conditions and the control 
condition was active student responding. Active student 
responding refers to when a student emits an observable, 
measureable response to an instructional antecedent, 
such as instructor presented questions (Heward, 1994). 
Research shows that the more actively engaged a 
student, the better their performance (e.g., Austin, Lee, & 
Carr, 2004; Austin et al., 2002; Greenwood, Delquadri, & 
Hall, 1984; Narayan et al, 1990; Neef, McCord, & Ferreri, 
2006; Randolph, 2007; Saville, etal., 2006). During the SRS 
condition students were actively responding to 
instructional antecedents, the questions. During the SSQ 
condition, although the participants did not make as 
many responses as during SRS, they had to remain on-task 
because they did not know when they would be called 
upon to answer a question and they had to respond when 


it was their turn to answer; whereas during the control 
condition, there was no contingency for the students to 
remain on task or engaged during the lecture. An 
instructor can improve student performance simply by 
increasing active student responding regardless of the 
mode of technology. As a case in point, Stowell and 
Nelson (2007) found no differences between SRS and 
response cards (a high tech and low tech version of 
active student responding). This leads us to believe that 
using technology for technology's sake does not improve 
instruction. There should be a specific purpose for 
effective use of technology, For example, an instructor 
may wish to receive immediate feedback on the 
accuracy of a large group of student responses. In this 
case, SRS would meet that goal. Another instructor may 
wish to increase the engagement of students in the class. 
In this case a further analysis of the type of engagement 
would reveal a better perspective of the type of high - 
tech or low-tech device that should be used. If the goal 
was to provide more practice a low-tech method such as 
response cards may be a more reasonable option than 
SRS. SRS may be preferable to other low-tech forms of 
active student responding when teaching a large group 
of students, immediate feedback on the precise average 
student performance is important, recording and scoring 
individual performances during class is desired, and 
privacy of students' responses is important (i.e., 
assessment). Response cards and choral responding 
may be appropriate when teaching small groups of 
students, presentation pace of questions is important 
(e.g., less than 30 second thinking time, short intertrial 
interval), privacy of student responses during the activity is 
not important (i.e., practice), funding for materials is 
limited, and instructor needs immediate feedback on an 
individual student performance or the general overall 
impression of group performance. However neither 
option provides for complex student responses. SRS, 
response cards and choral responding are appropriate 
for one correct short answer. Although SRS has been used 
to provoke student discussions (Judson & Sawada, 2002), 
the system itself does not allow for students to submit the 
discussions or independent thoughts via the clicker to be 
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scored, recorded, and have feedback provided. 

There are still many questions related to the efficacy of SRS 
in classrooms. For example, some types of SRS that allow 
the instructor to control the presentation of the question 
and length of thinking time and response time lends itself 
to investigations of instructional pacing. Instructional 
pacing incorporates instructor controlled elements such 
as simplicity or complexity of the question, time allotted 
for students to think about their answer, response prompts 
provided to students (if any), length of student response, 
time between the response and feedback, feedback 
itself, and the time between the end of feedback on one 
question and the presentation of another question 
(intertrial interval) (Tincani, et al., 2005) that make up a 
learning trial. For example, Robertson (2000) suggests 15- 
20 seconds per question for a group of 0 to 30 students, 30 
seconds per question for 30-100 students, 1 minute per 
question for 100 or more students. The pace in this study 
during the SRS condition was 30 seconds per questions 
with 23 participants. The optimal presentation rate of a 
learning trial through SRS is an empirical question that has 
not been addressed. 

Similarly, issues related to the presentation of questions 
should be investigated. For example, the manner in which 
questions are presented, interspersed throughout the 
lecture or massed presentation of questions, may impact 
student learning. Likewise, what is the optimum number of 
questions that should be presented during a lecture to 
improve student learning? In a review of SRS research 
Caldwell (2007) found that a range of two to five questions 
was typical for 50-minute class session. Flowever, eight to 
ten questions or more are presented during response 
card studies (Heward, 1994; Randolph, 2007). Perhaps 
once these empirical questions are addressed, more 
differential effects of SRS on student learning will be seen. 

In conclusion, the results of this study suggest, that no 
matter what form of technology one chooses to use from 
high to low tech, the most important variable is to get the 
students to make active responses in the classroom. 
Students will appreciate the teachers efforts and it will 
likely make the teachers to get more learning out of 
students. 
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